System for Synthetic Display of Multi-Modality Data

ABSTRACT

A computer-implemented method for providing a multi-modality visualization of a patient includes receiving one or more image datasets. Each image dataset corresponds to a distinct image modality. The image datasets are segmented into a plurality of anatomical objects. A list of clinical tasks associated with displaying the one or more image datasets are received. A machine learning model is used to determine visualization parameters for each anatomical object based on the list of clinical tasks. Then, a synthetic display of the image datasets is created by presenting each anatomical object according to its corresponding visualization parameters.

TECHNOLOGY FIELD

The present invention relates generally to methods, systems, andapparatuses for providing a synthetic display of multi-modality imagedata. The technology disclosed here may be used to perform avisualization/rendering of various modalities such as ComputedTomography (CT), Magnetic Resonance Imaging (MRI), Positron EmissionTomography (PET), and Ultrasound data.

BACKGROUND

Medical image modalities such as Computed Tomography (CT), MagneticResonance Imaging (MRI), and Ultrasound are capable of providing 3Dviews of a patient's anatomy. Increasingly, patients are scanned withmore than one modality in the course of treatment—such as both CT andMRI—or with more than one sub-type of a modality—such as bothT1-weighted and T2-weighted sequences for MRI. Some of the most recentlydeveloped medical imaging modalities provide for dual acquisition, suchas Dual Energy CT (DECT), Positron Emission Tomography CT (PET-CT) andPET-MR. Even with a single scan acquisition, there can be multiple waysof viewing the data, including with different reconstruction kernels orwith different visualization parameters (such as window/level).Physicians need these multiple acquisitions as different clinical tasksare better suited to—or in some cases require—the different modes ofimaging. However, reading through all the different acquisitionsavailable for a patient can be time consuming and thus expensive. Intoday's busy hospital or clinic, physicians need workflow aids thatpresent them the most relevant data in the most efficient manner.

To date, there has been limited work on combining information fromdifferent scans for efficient display. In some cases, two types of dataare combined with a complete overlay of the two images. The user isresponsible for controlling the blend function of how much of the baseimage is shown versus how much of the overlaid image. Similarly allmedical imaging workstations allow the user to do manual control ofwindow/level.

SUMMARY

Embodiments of the present invention address and overcome one or more ofthe above shortcomings and drawbacks, by providing methods, systems, andapparatuses that incorporate different types of multi-modality data intoa synthetic patient display that presents the most relevant andnecessary clinical information to the physician. As described in furtherdetail below, a machine learning-based approach is used to learn overtime correlations between clinical tasks and optimal presentation ofimaging data. Thus, integrated displays can be generated automaticallyor semi-automatically to present multi-modality imaging data in anoptimal manner for a given set of tasks.

According to some embodiments, a computer-implemented method forproviding a multi-modality visualization of a patient includes receivingone or more image datasets. Each image dataset corresponds to a distinctimage modality. The image datasets are segmented into a plurality ofanatomical objects. A list of clinical tasks associated with displayingthe one or more image datasets are received. A machine learning model isused to determine visualization parameters for each anatomical objectbased on the list of clinical tasks. Then, a synthetic display of theimage datasets is created by presenting each anatomical object accordingto its corresponding visualization parameters. For example, if thevisualization parameters correspond to colors, each anatomical objectmay be presented according to a distinct color.

In some embodiments, the aforementioned method further includesidentifying abnormalities in the anatomical objects depicted in theimage datasets. The machine learning model may then use theabnormalities to determine the visualization parameters. The machinelearning model may be trained offline by analyzing past radiologyreports to determine correspondences between past clinical tasksperformed by users and image modalities used to perform those pastclinical tasks.

Users can provide input in different embodiments to interact with thesynthetic display or the underlying data. For example, in someembodiments, the aforementioned method further includes receiving a userselection of a particular image modality and modifying the syntheticdisplay to present each anatomical object using the particular imagemodality. This user selection may be made via graphical user interfacecomponent that allows a user to select between a plurality of views suchas modality specific views (presenting all anatomical objects in aparticular image modality) and a blended view that presents anatomicalobjects in a mix of two or more image modalities. In some embodiments,user input is received associating the anatomical object with aparticular image modality. Based on this input, the synthetic displaycan be updated so that each anatomical object is displayed according tothe modality associated with the object. In some embodiments, the methodfurther includes detecting one or more changes to the synthetic displaymade by a user. Based on these changes and the list of clinical tasks,the machine learning model can be updated

A computer-implemented method for providing a multi-modalityvisualization of a patient includes analyzing one or more past radiologyreports to determine correspondences between past clinical tasksperformed by users and image modalities used to perform those pastclinical tasks. A machine learning model is trained based on thecorrespondences between past clinical tasks performed by users and imagemodalities used to perform those past clinical tasks. Next, one or moreimage datasets are received, with each image dataset corresponding to adistinct image modality. The machine learning model may then be used tocreate a synthetic display of a plurality of anatomical objects in theimage dataset suitable for performing one or more selected tasks. Thissynthetic display presents the anatomical objects using two or moreimaging modalities.

A system for providing a multi-modality visualization of a patientcomprising a monitor and a parallel processing platform. The parallelprocessing platform is configured to train a machine learning modelbased on the correspondences between past clinical tasks performed byusers and image modalities used to perform those past clinical tasks.The platform receives one or more image datasets and user selectedtasks. Each image dataset corresponding to a distinct image modality andcomprising a plurality of anatomical objects. The platform then uses themachine learning model to generate a set of visualization parameters foreach anatomical object suitable for performing the user selected tasks.A synthetic display of the anatomical objects may then be created on thedisplay, presenting each anatomical object according to itscorresponding set of visualization parameters.

Additional features and advantages of the invention will be madeapparent from the following detailed description of illustrativeembodiments that proceeds with reference to the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of the present invention are bestunderstood from the following detailed description when read inconnection with the accompanying drawings. For the purpose ofillustrating the invention, there is shown in the drawings embodimentsthat are presently preferred, it being understood, however, that theinvention is not limited to the specific instrumentalities disclosed.Included in the drawings are the following Figures:

FIG. 1 illustrates an example system which uses iGOR (intelligentGeneral Organ Recombination) to combine multiple input medical datasetsand produce an efficient visualization for clinicians, according to someembodiments;

FIG. 2 shows an example synthetic display showing the recombination oftwo different CT visualization settings following segmentation of thelung region in stage 1; and

FIG. 3 provides an example of a parallel processing memory architecturethat may be utilized to perform computations related to learning andvisualization operations performed by iGOR, according to someembodiments of the present invention.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The following disclosure describes the present invention according toseveral embodiments directed at methods, systems, and apparatusesrelated to the synthetic display of multi-modality data using atechnique referred to herein as intelligent General Organ Recombination,or “iGOR.” iGOR takes multiple medical imaging datasets (such as CT andMR) of a given patient as input, and combines them into a visualizedoutput; the specific output is based on predefined clinical tasks suchas liver cancer evaluation, lung cancer screening, cardiac screening,etc.

FIG. 1 illustrates an example system 100 which uses iGOR to combinemultiple input medical datasets and produce an efficient visualizationfor clinicians, according to some embodiments. Briefly, the process ispartitioned into two stages. In the first stage, datasets from multiplemodalities are automatically registered, landmarks (or abnormalities)are automatically detected, and the organs are automatically segmented.In the second stage, a learning algorithm is used to select the optimalway to combine and present the information to accomplish differentclinical tasks. The resulting image can recombine the organs fromdifferent imaging modalities to show, for example, the lungs as theylook in CT, with the liver as it looks in MR and the spleen as it looksin Ultrasound. The combined image is then presented on Display 140. ThisDisplay 140 could be a computer monitor or other equivalent device(e.g., smartphone or tablet screen). It should be noted that for thepurposes of the present description, FIG. 1 presents the Display 140 asblack and white; however, in practice, the Display 140 would utilizecolor to present a clearer and more obvious indication of thedifferences being observed.

The inputs to the system 100 include a plurality of MR Images 105A, aplurality of CT Images 105B, and a plurality of Ultrasound Images 105C.These image datasets can be received by the system 100, for example, byinteracting with one or more medical imaging scanners or extracting theimage datasets from a database of pre-generated images. In oneembodiment, the image datasets can be received by the system 100 viauser input (e.g., uploading via GUI).

As shown in FIG. 1, the first stage of the system contains modules forSegmentation 110A, Registration 110B, and Landmark Detection 110C. Thesemodules 110A, 110B, and 110C automatically register the different inputimages with each other, automatically detect anatomic landmarks withinthe images, and automatically segment the boundaries of the organs. Ingeneral, any technique known in the art can be used for implementing themodules and each module can be fully automated or may include somemanual input. Additionally, the order of application of the modules maybe defined to facilitate efficient processing of the images. Forexample, Landmark Detection 110C can be performed first, eitherautomatically or via a user selection of anatomical landmarks or otherlandmark points on each image. Then, Registration 110B can be performedbased on the landmarks. Similarly, Segmentation 110A may be performedbased on the landmarks, and possibly on the Registration 110Binformation as well.

It should be noted that the modules 110A, 110B, and 110C shown in thefirst stage of FIG. 1 are exemplary and additional or alternate modulesmay be used in different embodiments to provide different functionality.For example, in one embodiment, the landmark detection module isreplaced or augmented with automated detection of abnormalities such aslung nodules or liver lesions. In still another embodiment, theregistration matrix is the identity matrix because the differentmodalities are acquired simultaneously, as is the case for some types ofDECT, PET-CT and PET-MR.

The input images processed at stage 1 may also include functional imageswhich are derived from original images. These functional images mayinclude, for example, iodine maps or perfusion maps from DECT. However,functional images can also comprise feature maps or probability mapsderived from machine learning classifiers. This allows relevantinformation to be presented to the user when the derived images containinformation about cancer likelihood or other biomarkers. The methodpresented here allows a probability map of, for example, liver cancer tobe shown at the same time as a probability map for lung cancer in asingle volume, even when the probability maps are obtained by differentfunctions.

In the second stage of the system shown in FIG. 1, different ClinicalTasks 125 (i.e., Task1 through Task N) are selected as desired by theuser through a user interface (UI) 130. Example tasks include livercancer evaluation, lung cancer screening, cardiac screening, etc. In analternate embodiment, the clinical task is selected automatically basedon landmarks and/or abnormalities detected in Stage 1. For example, asystem to automatically detect lung nodules may be included in stage 1.If lung nodules are detected that are clinically relevant (based on sizeand other features), this triggers the lung screening task to beselected.

Based on the clinical task that was selected—either by the user orautomatically triggered—the input datasets that were registered,segmented and detected in stage 1 are combined and presented to theclinician to support the particular clinical task using iGOR 115. Asshown in FIG. 1, iGOR 115 includes a Learning Module 120 and aVisualization Module 135. The Learning Module 120 learns the parameterselection to combine and visualize the data according to trainingexamples. The Visualization Module 135 uses parameters provided by theLearning Module 120 to interface with the display computing devices inorder to provide the optimal display for the task.

Various types of learning may be applied by the Learning Module 120. Forexample, in some embodiments, the Learning Module is a neural networkthat uses the tasks and the results of the Stage 1 modules 110A, 110B,and 110C as inputs. Based on these inputs, the visualization parametersare determined. Deep learning or other more complex models could be usedin some embodiments to provide additional learning functionality to theLearning Module 120.

In some embodiments, the Learning Module 120 can operate in two modes.One mode is offline. In the offline mode, the Learning Module 120analyzes all user interactions and data inputs from a large set oftraining data. This training data may include, for example, radiologyreports; these reports are automatically analyzed to detect whatclinical task has been executed and what image modalities were used toperform that task. As an example, a radiology report may state thatT1-weighted MR images were read and that the liver was examined for thepresence of metastases. This serves as an example input that T1-weightedMR is an effective modality for looking at the liver if metastases are apossibility. Yet another report may state that sharp kernel CT was usedto examine the lungs for possible lung nodules. This, in turn, serves asan example input that sharp kernel CT is a good modality for lung cancerscreening. Training of the Learning Module 120 may be performed acrossmultiple clinical locations to provide a robust database for training.For example, training from multiple hospitals may be aggregated and usedfor training purposes. Alternatively, data from a single clinicallocation may be used for training to allow the Learning Module 120 toaccurately reflect the visualization preferences of that particularlocation. Moreover, in some embodiments, the Learning Module 120associates user identifiers (e.g., name, employee ID, etc.) with eachtask during learning to allow iGOR 115 to customize visualizations toparticular users.

Given a sufficiently large set of example data from radiology reports,the Learning Module 120 identifies coherent patterns of preferredimaging modalities for specific clinical tasks. The automatic dataregistration and segmentation computations in stage 1 are used topresent to the user a single image that shows—for example—the liver fromthe MR image at the same time as the lungs from the CT image. Eachregion can be displayed by the Visualization Module 135 (describedbelow) with different parameters (e.g., window/level, color maps, etc.).In another example, bones can be removed or enhanced on MR data based onsegmentation masks from CT data.

In other embodiments of the invention, the Learning Module 120 in stage2 operates in an online model, during which it does continuous learningand updates the learned setting as the system progressively collectsmore data from each new radiology report. In this way, the LearningModule 120 can learn hospital-specific preferences after being deployed.

The Visualization Module 135 uses the visualization parameters developedby the Learning Module 120 and uses it to present the image data on aDisplay 140. This presentation of image data is referred to herein as a“synthetic display” because it may combine the image data in a mannerthat is not a direct representation of a single image acquisition. TheVisualization Module 135 may be configured with the physicalspecifications of the Display 140, such as the screen size, aspectratio, resolution, color space, color depth, refresh rate, andinterfaces. If the physical specifications support the visualizationparameters provided by the Learning Module 120, then the VisualizationModule 135 may simply format the images and present them on the Display140. However, if the Display 140 does not support all of the specifiedvisualization parameters, the Visualization Module 135 may adjust theparameters based on the specifications of the Display 140. For example,if the visualization parameters specify that liver images should bedepicted in a particular color that is not available on the Display 140,the Visualization Module 135 may use the color space and/or color depthspecification values to select an alternative color that approximatesthe requested color.

In some embodiments, a Graphical User Interface (GUI) 130 allows theclinician to select between the different visualization methods providedby iGOR 115. In the example above with both MR and CT, the user may usea slider or other interface component to select between a pure MR view,a pure CT view, and a blended view of the different organs.Alternatively, the user may use radio buttons or other interfacecomponents to select which organs should be visualized in MR and whichin CT. Visualization can also be selected based on specific rules, suchas “show all areas where MR is hyper dense and CT signal is above100HU.”

In some embodiments, iGOR 115 can be further adapted to supportmulti-phase image visualization techniques, such as for medical imagingas may be utilized in hepatic perfusion visualization. Techniques formulti-phase image visualization are discussed in U.S. Pat. No. 8,755,635to Geiger et al., issued Jun. 17, 2014, entitled “Method and system fordata dependent multi phase visualization,” the entirety of which isincorporated herein by reference.

FIG. 2 shows an example synthetic display showing the recombination oftwo different CT visualization settings following segmentation of thelung region in stage 1. The task-specific setting allows simultaneousexamination of the lungs for possible nodules, and examination of thechest region for other abnormalities.

FIG. 3 provides an example of a parallel processing platform 300 thatmay be utilized to perform computations related to learning andvisualization operations performed by iGOR, according to someembodiments of the present invention. This platform 300 may be used inembodiments of the present invention where NVIDIA™ CUDA (or a similarparallel computing platform) is used. The architecture includes a hostcomputing unit (“host”) 305 and a graphics processing unit (GPU) device(“device”) 310 connected via a bus 315 (e.g., a PCIe bus). The host 305includes the central processing unit, or “CPU” (not shown in FIG. 3),and host memory 325 accessible to the CPU. The device 310 includes thegraphics processing unit (GPU) and its associated memory 320, referredto herein as device memory. The device memory 320 may include varioustypes of memory, each optimized for different memory usages. Forexample, in some embodiments, the device memory includes global memory,constant memory, and texture memory. The GUI 130 shown in FIG. 1 mayinterface with this parallel processing platform 300, either via adirect connection or via one or more computer networks.

Parallel portions of a deep learning application may be executed on theplatform 300 as “device kernels” or simply “kernels.” A kernel comprisesparameterized code configured to perform a particular function. Theparallel computing platform is configured to execute these kernels in anoptimal manner across the platform 300 based on parameters, settings,and other selections provided by the user. Additionally, in someembodiments, the parallel computing platform may include additionalfunctionalities to allow for automatic processing of kernels in anoptimal manner with minimal user inputs.

The processing required for each kernel is performed by grid of threadblocks (described in greater detail below). Using concurrent kernelexecution, streams, and synchronization with lightweight events, theplatform 300 of FIG. 3 (or similar architectures) may be used toparallelize training of a neural network such as the one applied byLearning Module 120.

The device 310 includes one or more thread blocks 330 which representthe computation unit of the device 310. The term thread block refers toa group of threads that can cooperate via shared memory and synchronizetheir execution to coordinate memory accesses. For example, in FIG. 3,threads 340, 345 and 350 operate in thread block 330 and access sharedmemory 335. Depending on the parallel computing platform used, threadblocks may be organized in a grid structure. A computation or series ofcomputations may then be mapped onto this grid. For example, inembodiments utilizing CUDA, computations may be mapped on one-, two-, orthree-dimensional grids. Each grid contains multiple thread blocks, andeach thread block contains multiple threads. For example, in FIG. 3, thethread blocks 330 are organized in a two dimensional grid structure withm+1 rows and n+1 columns. Generally, threads in different thread blocksof the same grid cannot communicate or synchronize with each other.However, thread blocks in the same grid can run on the samemultiprocessor within the GPU at the same time. The number of threads ineach thread block may be limited by hardware or software constraints. Toaddress this limitation, learning operations may be configured invarious manners to optimize use of the parallel computing platform. Forexample, in some embodiments, the operations of the Learning Module 120may be partitioned such that multiple kernels evaluate different tasks(e.g., Tasks 125) simultaneously in view of the available image dataprovided in Stage 1 (see FIG. 1). Alternatively, different input imagescould be evaluated in parallel across the entire task list.

Continuing with reference to FIG. 3, registers 355, 360, and 365represent the fast memory available to thread block 330. Each registeris only accessible by a single thread. Thus, for example, register 355may only be accessed by thread 340. Conversely, shared memory isallocated per thread block, so all threads in the block have access tothe same shared memory. Thus, shared memory 335 is designed to beaccessed, in parallel, by each thread 340, 345, and 350 in thread block330. Threads can access data in shared memory 335 loaded from devicememory 320 by other threads within the same thread block (e.g., threadblock 330). The device memory 320 is accessed by all blocks of the gridand may be implemented using, for example, Dynamic Random-Access Memory(DRAM).

Each thread can have one or more levels of memory access. For example,in the platform 300 of FIG. 3, each thread may have three levels ofmemory access. First, each thread 340, 345, 350, can read and write toits corresponding registers 355, 360, and 365. Registers provide thefastest memory access to threads because there are no synchronizationissues and the register is generally located close to a multiprocessorexecuting the thread. Second, each thread 340, 345, 350 in thread block330, may read and write data to the shared memory 335 corresponding tothat block 330. Generally, the time required for a thread to accessshared memory exceeds that of register access due to the need tosynchronize access among all the threads in the thread block. However,like the registers in the thread block, the shared memory is typicallylocated close to the multiprocessor executing the threads. The thirdlevel of memory access allows all threads on the device 310 to readand/or write to the device memory. Device memory requires the longesttime to access because access must be synchronized across the threadblocks operating on the device. For example, in some embodiments, theprocessing of each image or task at Stage 2 (see FIG. 1) is coded suchthat it primarily utilizes registers and shared memory and only utilizesdevice memory as necessary to move data in and out of a thread block.

The embodiments of the present disclosure may be implemented with anycombination of hardware and software. For example, aside from parallelprocessing architecture presented in FIG. 3, standard computingplatforms (e.g., servers, desktop computer, etc.) may be speciallyconfigured to perform the techniques discussed herein. In addition, theembodiments of the present disclosure may be included in an article ofmanufacture (e.g., one or more computer program products) having, forexample, computer-readable, non-transitory media. The media may haveembodied therein computer readable program code for providing andfacilitating the mechanisms of the embodiments of the presentdisclosure. The article of manufacture can be included as part of acomputer system or sold separately.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopeand spirit being indicated by the following claims.

An executable application, as used herein, comprises code or machinereadable instructions for conditioning the processor to implementpredetermined functions, such as those of an operating system, a contextdata acquisition system or other information processing system, forexample, in response to user command or input. An executable procedureis a segment of code or machine readable instruction, sub-routine, orother distinct section of code or portion of an executable applicationfor performing one or more particular processes. These processes mayinclude receiving input data and/or parameters, performing operations onreceived input data and/or performing functions in response to receivedinput parameters, and providing resulting output data and/or parameters.

A GUI, as used herein, comprises one or more display images, generatedby a display processor and enabling user interaction with a processor orother device and associated data acquisition and processing functions.The GUI also includes an executable procedure or executable application.The executable procedure or executable application conditions thedisplay processor to generate signals representing the GUI displayimages. These signals are supplied to a display device which displaysthe image for viewing by the user. The processor, under control of anexecutable procedure or executable application, manipulates the GUIdisplay images in response to signals received from the input devices.In this way, the user may interact with the display image using theinput devices, enabling user interaction with the processor or otherdevice.

The functions and process steps herein may be performed automatically orwholly or partially in response to user command. An activity (includinga step) performed automatically is performed in response to one or moreexecutable instructions or device operation without user directinitiation of the activity.

The system and processes of the figures are not exclusive. Othersystems, processes and menus may be derived in accordance with theprinciples of the invention to accomplish the same objectives. Althoughthis invention has been described with reference to particularembodiments, it is to be understood that the embodiments and variationsshown and described herein are for illustration purposes only.Modifications to the current design may be implemented by those skilledin the art, without departing from the scope of the invention. Asdescribed herein, the various systems, subsystems, agents, managers andprocesses can be implemented using hardware components, softwarecomponents, and/or combinations thereof. No claim element herein is tobe construed under the provisions of 35 U.S.C. 112, sixth paragraph,unless the element is expressly recited using the phrase “means for.”

We claim:
 1. A computer-implemented method for providing amulti-modality visualization of a patient, the method comprising:receiving one or more image datasets, each image dataset correspondingto a distinct image modality; segmenting the one or more image datasetsinto a plurality of anatomical objects; receiving a list of clinicaltasks associated with displaying the one or more image datasets; foreach anatomical object, using a machine learning model to determinevisualization parameters for the anatomical object based on the list ofclinical tasks; and creating a synthetic display of the one or moreimage datasets by presenting each anatomical object according to itscorresponding visualization parameters.
 2. The method of claim 1,further comprising: identifying one or more landmarks in the one or moreimage datasets; and performing registration of the one or more imagedatasets to yield registered image data, wherein the machine learningmodel uses the one or more of (i) the landmarks and (ii) the registeredimage data to determine the visualization parameters.
 3. The method ofclaim 2, wherein the registered image data comprises an identity matrix.4. The method of claim 1, further comprising: identifying abnormalitiesin the plurality of anatomical objects depicted in the one or more imagedatasets, wherein the machine learning model uses the abnormalities todetermine the visualization parameters.
 5. The method of claim 1,wherein the visualization parameters correspond to colors and eachanatomical object is presented in the synthetic display according to adistinct color.
 6. The method of claim 1, further comprising: receivinga user selection of a particular image modality; and modifying thesynthetic display of the one or more image datasets to present eachanatomical object using the particular image modality.
 7. The method ofclaim 6, wherein the user selection is made via graphical user interfacecomponent that allows a user to select between a plurality of viewscomprising: a plurality of modality specific views, each modalityspecific view presenting all anatomical objects in a particular imagemodality, and a blended view that presents anatomical objects in a mixof two or more image modalities.
 8. The method of claim 1, furthercomprising: for each anatomical object, receiving user input associatingthe anatomical object with a particular image modality; and presentingeach anatomical object in the synthetic display according to the imagemodality associated with the anatomical object.
 9. The method of claim1, wherein the machine learning model is trained offline by analyzingone or more past radiology reports to determine correspondences betweenpast clinical tasks performed by users and image modalities used toperform those past clinical tasks.
 10. The method of claim 1, furthercomprising: after creating the synthetic display, detecting one or morechanges to the synthetic display made by a user; and updating themachine learning model based on the one or more changes and the list ofclinical tasks.
 11. A computer-implemented method for providing amulti-modality visualization of a patient, the method comprising:analyzing one or more past radiology reports to determinecorrespondences between past clinical tasks performed by users and imagemodalities used to perform those past clinical tasks; training a machinelearning model based on the correspondences between past clinical tasksperformed by users and image modalities used to perform those pastclinical tasks; receiving one or more image datasets, each image datasetcorresponding to a distinct image modality; and using the machinelearning model to create a synthetic display of a plurality ofanatomical objects in the image dataset suitable for performing one ormore selected tasks, wherein the synthetic display presents theanatomical objects using two or more imaging modalities.
 12. The methodof claim 11, further comprising: identifying one or more landmarks inthe one or more image datasets; and performing registration of the oneor more image datasets to yield registered image data, wherein themachine learning model uses one or more of the landmarks and theregistered image data to determine visualization parameters for creatingthe synthetic display.
 13. The method of claim 11, further comprising:identifying abnormalities in the plurality of anatomical objectsdepicted in the one or more image datasets, wherein the machine learningmodel selects the two or more imaging modalities based on theabnormalities.
 14. The method of claim 11, wherein each anatomicalobject is presented in the synthetic display according to a distinctcolor.
 15. The method of claim 11, further comprising: receiving a userselection of a particular image modality; and modifying the syntheticdisplay to present each anatomical object using the particular imagemodality.
 16. The method of claim 15, wherein the user selection is madevia graphical user interface component that allows a user to selectbetween a plurality of views comprising: a plurality of modalityspecific views, each modality specific view presenting all anatomicalobjects in a particular image modality, and a blended view that presentsanatomical objects in a mix of two or more image modalities.
 17. Themethod of claim 11, further comprising: for each anatomical object,receiving user input associating the anatomical object with a particularimage modality; and presenting each anatomical object in the syntheticdisplay according to the image modality associated with the anatomicalobject.
 18. The method of claim 11, wherein the machine learning modelis trained offline by analyzing one or more past radiology reports todetermine correspondences between the past clinical tasks performed byusers and image modalities used to perform those past clinical tasks.19. The method of claim 11, further comprising: after creating thesynthetic display, detecting one or more changes to the syntheticdisplay made by a user; and updating the machine learning model based onthe one or more changes and the list of clinical tasks.
 20. A system forproviding a multi-modality visualization of a patient, the systemcomprising: a monitor; and a parallel processing platform configured to:train a machine learning model based on the correspondences between pastclinical tasks performed by users and image modalities used to performthose past clinical tasks; receive one or more image datasets, eachimage dataset corresponding to a distinct image modality and comprisinga plurality of anatomical objects; receiving one or more user selectedtasks; use the machine learning model to generate a set of visualizationparameters for each anatomical object suitable for performing the one ormore user selected tasks; and create a synthetic display of theplurality of anatomical objects on the display, wherein the syntheticdisplay presents the anatomical object according to its correspondingset of visualization parameters.